QaNote
Testing Mindset: Testing is the process of executing a program with the intent of finding errors. =Psychology= * Testing is the process of executing a program with the intent of finding errors ** prove the existence of defect =Test Types= * Integration tests, ...different units of work are combined into a workflow. * Functional tests, corresponds to testing application use cases. * Stress and load tests, ...how well will the application perform when many people are using it at once. * Acceptance tests, ...the application must also meet the customer’s needs. acceptance{ stress/load{ functional{ integration{ unit; } // out of integration } // out of functional } // out of stress/load } // out of acceptance * Unit Testing Three flavors of unit tests: logic, integration, and functional. functional unit test{ integration unit test{ logic unit test; } // out of integration } ; Logic unit test : A test that exercises code by focusing on a single method. You can control the boundaries of a given test method using mock objects or stubs. ; Integration unit test : A test that focuses on the interaction between components in their real environment (or part of the real environment). For example, code that accesses a database has tests that effectively call the database ; Functional unit test : A test that extends the boundaries of integration unit testing to confirm a stimulus response. =Testing Techniques= * Black Box Test *# Functionality Testing *# GUI Testing *# Performance Testing *# Stress and Load Testing *# Compatibility Test * White Box Testing *# Code Coverage *# Path Coverage *# Code Analysis * Manual Test * Automation Test *# Development Process Study *# Testing Tools Selection *# Implementation =Strategies= * Black-Box Testing ** exhaustive input testing (impractical) ** unconcerned about the internal behaviour and structure; ** concentrate on finding circumstances in which the program does not behave according to its specifications. * White-Box Testing ** aka: logic-driven testing (may NOT cover the specification) ** exhaustive path testing (impractical, missing critical path, insufficient without exhaustive input test) Test Design # Identifying Equivalence Classes - EC ## If the input condition is a range of values, ##* identify one valid EC (between) and ##* two invalid ECs (lower and greater); ## If the input condition specifies a set of input values and there is reason to believe that the program handles each one differently, ##* identify a valid EC for each input and ##* one invalid EC. ## If the input condition specifies a 'must be' situation, ##* identify one valid EC and ##* one invalid EC. ## If there is reason to believe that the program does not handle elements in an EC identically, split the EC into smaller ECs. # Defining the test cases ## Assign a unique identifier to each EC. ## Write test cases to cover all the valid ECs. A single test case can cover more than one EC. ## Write test cases to cover all the invalid ECs. A single test case can cover one and only one invalid EC. Black Box # Equivalence Partitioning; # Boundary-value analysis; # Cause-effect graphing; # Error guessing. White Box # Statement coverage; # Decision coverage; # Condition coverage; # Decision/condition coverage; # Multiple-condition coverage. Refer to test case design =Testing Principles= # A test case must include a definition of the expected output or result. ## An explicit description of the input ## A precise description of the correct output for that set of input # A programmer should not test his/her own program. # A programming organization should not test its own programs. Refer to basic principles =Full Life cycle= # Identify Requirements # Acceptance Criteria # Strategy and Design # Planning # Executing # Defect Tracking and Bug Fix # Review and Audit # Acceptance and Baseline # Identify Requirements # ... =Testing Process= * V-Model open * Defect prevention *# Detect the abnormality. *# Stop what you’re doing. *# Fix or correct the immediate problem. *# Investigate the root cause and install a countermeasure. =Methodologies= * Agile click * Waterfall click =Agile Testing Challenges= * Team may not value testers; * Testers may not value team; * Unclear role of testers on the team; * Testing is often squeezed to the deadlines; * Developers and tester are often in different operational silos; * Team may not have the skills/domain expertise/time/attention to develop/execute test cases effectively. Reference click =Automation Testing= Concepts * TDD * BDD / DDD * CDT ** James Bash ** Context-Driven-Testing Automation Frameworks Why automation? * Provides safety net * Supports rapid iterations * Provides footholds to keep notching upward * Provides rapid feedback * Focuses effort on what is valuable * Frees people to do their best work TDD Test Driven Development ; Test Driven Development / Test First Development * JUnit * TestNG ; Parallel * Run JUnit in parallel click * Maven fork option click Web Testing ; Links * Random name generator fakename listofrand-name * Selenium Note ** s free and open source ** have a large user base and helping communities ** have cross Browser compatibility ** have great platform compatibility ** supports multiple programming languages ** has fresh and regular repository developments ** supports distributed testing * soapUI * Browsers ** Firefox + XPath Checker (Right click -> View XPath) ** Chrome + XPath Helper (X + Shift + Ctrl) Mobile Testing * Appium ** click ** dmg * SauceLab ** sauce-lab-java * nativedriver click See also: xamarin, appium Android Testing ; What to test # Activity Testing # Content Provider Testing # Service Testing ; How to test * UiAminator click ** By api Espresso * guide * kit * guide * cheat-sheet * configure and guideline * ViewMatcher api withId() hamcrest cheat sheet * Espresso api onView() * Setup and launch click * ViewActions api click() * Advanced Android Espresso click/slide Instrumentation * Instrumentation api * seledroid click * uiautomator click ** android-uiautomator-server click ** android-uiautomator-json-rpc-server click ** python wrapper uiautomator click * trade federation click See also * google testing-support-lib click * top-5 frameworks click ** robotium solo ext-solo ** robolectric click * monkeyrunner click * Android ChromeDriver click iOS Testing * iOS testing guide click * iOS click * ios-driver click Cloud Testing * SauceLab java+mobile+saucelab BDD Behavior Driven Development * Cucumber Ruby ** Cucumber Learning Note incomplete ** Watir / Watij * Cucumber-jvm ** Cucumber-jvm Learning Note in-progress ** Parallel click ** IBM Automated testing with Selenium and Cucumber click * RSpec * Concordion (readable bdd) * fitnesse (wiki to bdd cases) * JBehave ** jBehave Learning Note Defect Detection Tbd Defect Prevention Tbd Refer software-defect-prevention-nutshell =Test Management= TestLink * Get Started with TestLink click ** TestLink requires httpd server, Apacheloungh click ** TestLink and LDAP click * How to Update TestLink Test Case Execution Status Remotely Through Selenium click Zephyr * REST API click click jira-rest ** Create test step click * Zephyr SOAP API v4.6 =Continuous Integration= * Continuous Integration * Jenkins =Mobile Test= Likely Failure Criteria # You are using indexes in too many places so as to identify elements in your screen. # You are using Image recognition or OCR extensively and not for content verification. # When running the same script on a device with different screen sizes, it never works on the first try. # Scrolling the device screen is coordinate-based and very hard to stabilize. # You are wasting more time in stabilizing your tests than on building them. # You feel frustrated every time you open the test environment due to its slowness and unpredictability # You are working for several months but only have a dozen tests # You can’t remember the last time that all the tests ran successfully. # You aren't controlling all the parts of your automation environment such as devices and versions # You are masking the project status from your managers. # You don’t have a process where the tests run 24/7. # Deploying the build production (IPA / APK) from the R&D is a manual process. Refer to https://selenium2automate.wordpress.com/tag/symptoms/ =Oogler Testing= Roles * SWE - Software Engineer * SET - Software Engineer in Test * TE - Test Engineer SET/TE who can programs effectively in multiple languages. Responsibilities * SWE write test code * SET - write test infrastructure * TE - execute, track, coordinate and ... the tests. =Certificate= ISTQB * http://swtestengineers.blogspot.com.au/2012/04/full-study-material-for-istqb.html * http://istqbexamcertification.com/category/fundamentals-of-testing/ =Blogs= * Pairing With Developers: A Guide For Testers click